106 research outputs found
Semantic Robot Programming for Goal-Directed Manipulation in Cluttered Scenes
We present the Semantic Robot Programming (SRP) paradigm as a convergence of
robot programming by demonstration and semantic mapping. In SRP, a user can
directly program a robot manipulator by demonstrating a snapshot of their
intended goal scene in workspace. The robot then parses this goal as a scene
graph comprised of object poses and inter-object relations, assuming known
object geometries. Task and motion planning is then used to realize the user's
goal from an arbitrary initial scene configuration. Even when faced with
different initial scene configurations, SRP enables the robot to seamlessly
adapt to reach the user's demonstrated goal. For scene perception, we propose
the Discriminatively-Informed Generative Estimation of Scenes and Transforms
(DIGEST) method to infer the initial and goal states of the world from RGBD
images. The efficacy of SRP with DIGEST perception is demonstrated for the task
of tray-setting with a Michigan Progress Fetch robot. Scene perception and task
execution are evaluated with a public household occlusion dataset and our
cluttered scene dataset.Comment: published in ICRA 201
NARF22: Neural Articulated Radiance Fields for Configuration-Aware Rendering
Articulated objects pose a unique challenge for robotic perception and
manipulation. Their increased number of degrees-of-freedom makes tasks such as
localization computationally difficult, while also making the process of
real-world dataset collection unscalable. With the aim of addressing these
scalability issues, we propose Neural Articulated Radiance Fields (NARF22), a
pipeline which uses a fully-differentiable, configuration-parameterized Neural
Radiance Field (NeRF) as a means of providing high quality renderings of
articulated objects. NARF22 requires no explicit knowledge of the object
structure at inference time. We propose a two-stage parts-based training
mechanism which allows the object rendering models to generalize well across
the configuration space even if the underlying training data has as few as one
configuration represented. We demonstrate the efficacy of NARF22 by training
configurable renderers on a real-world articulated tool dataset collected via a
Fetch mobile manipulation robot. We show the applicability of the model to
gradient-based inference methods through a configuration estimation and 6
degree-of-freedom pose refinement task. The project webpage is available at:
https://progress.eecs.umich.edu/projects/narf/.Comment: Accepted to the 2022 IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS). Contact: Stanley Lewis, [email protected]
- …